Personal identity authenticatication process and system

ABSTRACT

A personal identity authentication process and system use a class specific linear discriminant transformation to test authenticity of a probe face image. A ‘client acceptance’ approach, an ‘imposter rejection’ approach and a ‘fused’ approach are described.

[0001] This invention relates to a personal identity authenticationprocess and system.

[0002] The problem of computerised personal identity authentication hasreceived considerable attention over recent years, finding numerouscommercial and law enforcement applications.

[0003] By way of illustration, these include the verification of creditcards, passports, driver's licences and the like, matching of controlledphotographs such as mug shots, recognition of suspects from CCTV videoagainst a database of known face images and control of access tobuildings and teleservices, such as bank teller machines.

[0004] A paper entitled “Face Recognition: Features versus Templates” byR Brunelli and T Poggio in IEEE trans on PAMI, Vol 15 pp 1042-1052, 1993presents a comparison of two basic approaches; namely, ageometric-feature based approach and a template or statistical-featurematching approach, the latter being favoured by the authors.

[0005] The most commonly used statistical representation for faceauthentication is the Karhunen-Loeve expansion, known also as PrincipalComponent Analysis (PCA), by which face images are represented in alower-dimensional sub-space using PCA bases defined by eigenvectors,often referred to as ‘eigenfaces’.

[0006] Although this approach provides a very efficient means of datacompression, it does not guarantee the most efficient compression ofdiscriminatory information.

[0007] More recently, the technique of linear discriminant analysis(LDA) has been adapted to the problem of face recognition. Again, theface images are represented in a lower dimensional sub-space, but usingLDA bases defined by eigenvectors which are often referred to as ‘fisherfaces’, and these have been demonstrated to far outperform the PCArepresentation using ‘eigenfaces’. However, conventional LDArepresentations involve use of multiple, shared ‘fisher faces’,necessitating complex and computationally intensive matrix operations,and this presents a significant technical problem in terms ofperformance, processing speed and ease of adding to, or updating thedatabase of face images against which a probe image is tested.

[0008] According to a first aspect of the invention there is provided apersonal identity authentication process comprising the steps of usinglinear discriminant analysis (LDA) to derive a class-specific lineardiscriminant transformation a, from N vectors z_(j) (j=1,2 . . . N)defining respective training images, there being m different classes ofsaid training images with the i^(th) class ω_(i) containing a respectivenumber N_(i) of said training images such that${N = {\sum\limits_{i = 1}^{m}\quad N_{i}}},$

[0009] projecting a vector z_(p) defining a probe image onto saidclass-specific linear discriminant transformation a_(i), comparing theprojected vector a_(i)^(T)z_(p)

[0010] with a reference vector for the i^(th) class ω_(i) and evaluatingauthenticity of the probe image with respect to the i^(th) class independence on the comparison.

[0011] According to another aspect of the invention there is provided apersonal identity authentication system comprising data storage and dataprocessing means for carrying out the process defined in accordance withsaid first aspect of the invention.

[0012] According to a third aspect of the invention there is provided apersonal identity authentication system for evaluating authenticity of aprobe image with respect to one or more of m different classes oftraining images, wherein said training images are defined by respectivevectors z_(j) (j=1,2 . . . N), there being a total of N vectors z_(j),and the number of training images in the i^(th) class ω_(i) is N_(i)such that ${N = {\sum\limits_{i = 1}^{m}\quad N_{i}}},$

[0013] the personal identity authentication system comprising datastorage means for storing a class-specific linear discriminant a_(i), asdefined in accordance with said first aspect of the invention, for eachof said classes ω_(i) (i=1,2 . . . m), and data processing means foraccessing a said class-specific linear discriminant transformation a_(i)from said data storage means, projecting a vector z_(p) defining saidprobe image onto the accessed class-specific linear discriminanttransformation a_(i), comparing the projected vector a_(i)^(T)z_(p)

[0014] with a reference vector for the i^(th) class ω_(i) and evaluatingauthenticity of the probe image with respect to the i^(th) class ω_(i)in dependence on the comparison.

[0015] According to yet a further aspect of the invention there isprovided a computer readable medium containing computer executableinstructions for carrying out a process as defined in accordance withsaid first aspect of the invention.

[0016] An embodiment of the invention will now be described, by way ofexample only, with reference to the accompanying drawings of which:

[0017]FIG. 1a shows a histogram of a test statistic t_(i) for projectedimposter face images,

[0018]FIG. 1b shows a histogram of the same test statistic for projectedclient face images, and

[0019]FIG. 2 is a schematic representation of a semi-centralisedpersonal identity authentication system according to the invention.

[0020] In this embodiment of the invention, it will be assumed thatthere is a total of N training images representing m differentindividuals, referred to herein as ‘clients’. The total number oftraining images N is given by the expression:${N = {\sum\limits_{i = 1}^{m}\quad N_{i}}},$

[0021] where N_(i) is the number of training images representing thei^(th) client, defining a distinct client class ω_(i). The number N_(i)of training images need not be the same for each client class.Typically, N might be of the order of 10³ and m might be of the order10².

[0022] The training images are derived from some biometric data, assumedto be appropriately registered and normalised photometrically.

[0023] In this embodiment, frontal face images are used; however, imagesof other biometric data could alternatively be used—e.g. profile facialimages.

[0024] As will be explained, the described personal identityauthentication process and system can be used to evaluate authenticityof a probe face image presented as being that of one of the clients,either accepting or rejecting the claimed identity. This process findsapplication, inter alia, in the verification of credit cards, passports,driver's licences and the like.

[0025] Each training face image is defined by a two-dimensional D×Darray of grey level intensity values which can be thought of as ad-dimensional vector z, where d=D². Typically, d may be of the order of2¹⁴ and an ensemble of face images will map into a collection of pointswithin this huge d-dimensional space. Face images, being similar inoverall configuration, will not be randomly distributed within thisspace, and thus can be defined in terms of a relatively low-dimensionalsub-space. Conventionally, the d-dimensional vectors z are projectedinto a lower dimensional sub-space spanned by the training face images,and this is accomplished using a PCA projection matrix U generated usingthe aforementioned Principal Component Analysis (PCA).

[0026] The projection matrix U is derived from the mixture covariancematrix Σ, given by the expression: $\begin{matrix}{{\sum{= {\sum\limits_{j = 1}^{N}{\left( {z_{j} - \mu} \right)\left( {z_{j} - \mu} \right)^{T}}}}},} & (1)\end{matrix}$

[0027] where z_(j) is the d-dimensional vector defining the j^(th)training face image and μ is the global mean vector given by theexpression: $\mu = {\sum\limits_{j = 1}^{N}z_{j}}$

[0028] If, as here, the dimensionality d of the image vectors z islarger than the number of training images N, the mixture covariancematrix Σ will have n≦N non-zero eigenvalues. The respective eigenvectorsu₁,u₂ . . . u_(n) associated with these non-zero eigenvalues (ranked inorder of decreasing size) define the PCA bases used to represent thesub-space spanned by the training face images and, to this end, are usedto construct the PCA projection matrix U, which takes the form

U=[u₁,u₂ . . . u_(n)].

[0029] As described in a paper entitled “Introduction to StatisticalPattern Recognition” by K Fukunaga, Academic Press, New York, 1990 theeigenvalue analysis of the mixture covariance matrix Σ may, for reasonsof computational convenience, be carried out in a sub-space of dimensiond′, where d′<d.

[0030] Each eigenvector u is of dimension d and will be representativeof an image having a face-like appearance, resembling the face imagesfrom which it is derived. It is for this reason that the eigenvectors uare sometimes referred to as “eigenfaces”.

[0031] Having obtained the PCA projection matrix U, the N vectors z_(j)(j=1,2 . . . N) defining the training face images are projected, aftercentralisation, into the lower-dimensional sub-space spanned by theeigenvectors u₁,u₂ . . . U_(n) to generate N corresponding n-dimensionalvectors x_(j), given by the expression:

x _(j) =U ^(T)(z _(j)−μ) for j=1,2 . . . N  (2)

[0032] At this stage, it has been hitherto customary to apply lineardiscriminant analysis (LDA). The LDA bases used to represent thesub-space spanned by the vectors x_(j) are defined by m−1 eigenvectorsv₁, v₂ . . . v_(m−1) associated with the non-zero eigenvalues of amatrix Φ⁻¹S_(B), where Φ is the mixture covariance matrix of the vectorsx_(j), given by the expression: $\begin{matrix}{\Phi = {\frac{1}{N}{\sum\limits_{j = 1}^{N}{x_{j}x_{j}^{T}}}}} & (3)\end{matrix}$

[0033] and S_(B) is the between-class scatter matrix derived from themean ν_(i) of the projected vectors x in each said client class ω, wherei=1,2,3 . . . m, and ν_(i) is given by the expression: $\begin{matrix}{v_{i} = {{\frac{1}{N}{\sum\limits_{j = 1}^{N_{i}}{x_{j}\quad {for}\quad {all}\quad x_{j}}}} \in \omega_{i}}} & (4)\end{matrix}$

[0034] Again, each eigenvector v is representative of an image having asomewhat face-like appearance and is sometimes referred to as a ‘fisherface’, and these vectors are used to construct a LDA projection matrixV=[v₁,v₂ . . . v_(m−1)]. Adopting the conventional approach, a vectorz_(p) defining a probe face image; that is, a face image presented asbeing that of one of the m clients whose authenticity is to beevaluated, is initially projected into the n-dimensional sub-spacedefined by the ‘eigenfaces’ of the PCA projection matrix U and is thenprojected into the m−1 dimensional sub-space defined by the ‘fisherfaces’ of the LDA projection matrix V to generated a projected vectory_(p) given by the expression:

y _(p) =V ^(T) U ^(T)(z _(p)−μ)

[0035] Verification, or otherwise, of the claimed identity is thencarried out by testing the projected vector y_(p) against a projectedmean γ_(i) for the relevant client class ω_(i), where

γ_(i) =V ^(T) v _(i)

[0036] Then, if the projected vector y is within a predetermineddistance of the projected mean γ_(i) authenticity of the probe image isaccepted as being that of the i^(th) client (i.e. the claimed identityis accepted); otherwise, authenticity of the probe image is rejected asbeing that of an imposter (i.e. the claimed identity is rejected).

[0037] Inspection of Equation 5 above shows that the conventionalcomputational process involves multiple shared ‘fisher faces’represented by the eigenvectors v₁,v₂ . . . v_(m−1) defining theprojection matrix V and is always the same regardless of the clientclass ω_(i) against which a probe face image is being tested. Thisapproach is computationally intensive involving complex matrixoperations and therefore generally unsatisfactory.

[0038] In contrast to this conventional approach, which requires theprocessing of multiple shared ‘fisher faces’, the present inventionadopts an entirely different approach which involves processing asingle, class-specific ‘fisher face’ defined by a one-dimensional lineardiscriminant transformation. This approach avoids the use of multiple,shared fisher faces giving a considerable saving in computationalcomplexity. To this end, the personal identity authentication process isredefined in terms of a two-class problem; that is, the client classω_(i) containing the N_(i) training face images of the i^(th) client andan imposter class Ω_(i) based on the N−N_(i) remaining training faceimages. Clearly, there will be a client class ω_(i) and an associatedimposter class Ω_(i) for each of the m clients (i=1,2 . . . m).

[0039] With this formulation, the mean v_(Ω) _(i) of the projectedvectors x for the imposter class Ω_(i) can be expressed as:$\begin{matrix}{{v_{\Omega_{i}} = {{\frac{1}{N - N_{i}}{\sum\limits_{j = 1}^{N - N_{i}}{x_{j}\quad {for}\quad {all}\quad x_{j}}}} \notin \omega_{i}}},} & (7)\end{matrix}$

[0040] which, by comparison with Equation 4 above, can be expressed interms of ν_(j) as $\begin{matrix}{v_{\Omega_{i}} = {{- \frac{N}{N - N_{i}}}v_{i}}} & (8)\end{matrix}$

[0041] Thus, the mean of the i^(th) imposter class Ω_(i) is shifted inthe opposite direction to the mean of the i^(th) client class ω_(i), themagnitude of the shift being given by the ratio of the respectivenumbers of training face images N_(i), N−N_(i) in the two classes. Thisratio will normally be small and so the mean of the imposter class Ω_(i)will stay close to the origin irrespective of the client class ω_(i)against which a probe face image is being tested.

[0042] The between-class scatter matrix M_(i) for the two classesω_(i),Ω_(i) can be expressed as: $\begin{matrix}{{M_{i} = {{\frac{N - N_{i}}{N}\left( \frac{N_{i}}{N - N_{i}} \right)^{2}v_{i}v_{i}^{T}} + {\frac{N_{i}}{N}v_{i}v_{i}^{T}}}},} & (9)\end{matrix}$

[0043] which can be reduced to $\begin{matrix}{M_{i} = {\frac{N_{i}}{N - N_{i}}v_{i}v_{i}^{T}}} & (10)\end{matrix}$

[0044] Also, the covariance matrix Φ_(Ω) of the imposter class Ω_(i)estimated as: $\begin{matrix}{\Phi_{\Omega} = {{\frac{1}{N - N_{i}}{\sum\limits_{j = 1}^{N - N_{i}}{\left( {x_{j} + {\frac{N_{i}}{N - N_{i}}v_{i}}} \right)\left( {x_{j} + {\frac{N_{i}}{N - N_{i}}v_{i}}} \right)^{T}\quad x_{j}}}} \in \omega_{i}}} & (11)\end{matrix}$

[0045] can be expressed in terms of the mixture covariance matrix Φ byrewriting equation 11 above as: $\begin{matrix}\begin{matrix}{\Phi_{\Omega} = {\frac{1}{N - N_{i}}\left\lbrack {{\sum\limits_{j = 1}^{N}{\left( {x_{j} + {\frac{N_{i}}{N - N_{i}}v_{i}}} \right)\left( {x_{j} + {\frac{N_{i}}{N - N_{i}}v_{i}}} \right)^{T}}} -} \right.}} \\\left. {\sum\limits_{j = 1}^{N_{i}}{\left( {x_{j} + {\frac{N_{i}}{N - N_{i}}v_{i}}} \right)\left( {x_{j} + {\frac{N_{i}}{N - N_{i}}v_{i}}} \right)^{T}}} \right\rbrack\end{matrix} & (12)\end{matrix}$

[0046] where the vectors in the second sum belong to the client class.In fact, the second sum is related to the covariance matrix Φ_(i) forthe client class ω_(i), i.e. $\begin{matrix}\begin{matrix}{\Phi_{i} = {\frac{1}{N_{i}}{\sum\limits_{j = 1}^{N_{i}}{\left( {x_{j} - v_{i}} \right)\left( {x_{j} - v_{i}} \right)^{T}}}}} & {\quad {x_{j} \in \omega_{i}}}\end{matrix} & (13) \\{as} & \quad \\{{\sum\limits_{j = 1}^{N_{i}}{\left( {x_{j} + {\frac{N_{i}}{N - N_{i}}v_{i}}} \right)\left( {x_{j} + {\frac{N_{i}}{N - N_{i}}v_{i}}} \right)^{T}}} = {N_{i}\left\lbrack {\Phi_{i} + {\left( \frac{N}{N - N_{i}} \right)^{2}v_{i}v_{i}^{T}}} \right\rbrack}} & (14)\end{matrix}$

[0047] as

[0048] Thus, simplifying Equation 12 above, it can be shown that:$\begin{matrix}{\Phi_{\Omega} = {\frac{1}{N - N_{i}}\left\lbrack {{N\quad \Phi} - {N_{i}\Phi_{i}} - {\frac{{NN}_{i}}{N - N_{i}}v_{i}v_{i}^{T}}} \right\rbrack}} & (15)\end{matrix}$

[0049] The within-class scatter matrix Σ_(i) for the it client class isnow obtained by a weighted averaging of the covariance matrices of theimposter and client classes i.e. $\begin{matrix}{\sum\limits_{i}{= {{\frac{N - N_{i}}{N}\Phi_{\Omega}} + {\frac{N_{i}}{N}\Phi_{i}}}}} & (16)\end{matrix}$

[0050] and by substituting from Equation 15 above, and simplifying, itcan be shown that $\begin{matrix}{\sum\limits_{i}{= {\Phi - M_{i}}}} & (17)\end{matrix}$

[0051] A class specific linear discriminant transformation a_(i) forthis two class problem can be obtained from the eigenvectors of matrix$\sum\limits_{i}^{- 1}M_{i}$

[0052] associated with nonzero eigenvalues. In fact, in this two classproblem there is only one such eigenvector v_(i) that satisfies theequation $\begin{matrix}{{{\sum\limits_{i}^{- 1}{M_{i}v_{i}}} - {\lambda \quad v_{i}}} = 0} & (18)\end{matrix}$

[0053] with λ≠0 provided ν_(i) is non zero. As there is only onesolution to the eigenvalue problem it can be easily shown that theeigenvector v_(i) can be found directly, without performing anyeigenanalysis, as $\begin{matrix}{v_{i} = {\sum\limits_{i}^{- 1}v_{i}}} & (19)\end{matrix}$

[0054] This becomes apparent by substituting for v_(i) in Equations 18and 19 above and for M_(i) from Equation 10 above, i.e. $\begin{matrix}{{{\frac{N_{i}}{N - N_{i}}{\sum\limits_{i}^{- 1}{v_{i}v_{i}^{T}{\sum\limits_{i}^{- 1}v_{i}}}}} = {\lambda \quad v_{i}}},} & (20)\end{matrix}$

[0055] which also shows that the eigenvalue λ is given by$\begin{matrix}{\lambda = {\frac{N_{i}}{N - N_{i}}v_{i}^{T}{\sum\limits_{i}^{- 1}v_{i}}}} & (21)\end{matrix}$

[0056] The eigenvector v_(i) is used as the base for a lineardiscriminant transformation a_(i) for the i^(th) client class ω_(i)given by the expression:

a_(i)=U v_(i),  (22)

[0057] and it is this transformation as that defines a one-dimensional,class-specific ‘fisher face’ used to test the authenticity of a probeimage face in accordance with the present invention.

[0058] In one approach, referred to herein as “the client acceptance”approach, a vector z_(p) defining a probe image face is projected ontothe class specific ‘fisher face’ using the transformation a_(i) and theprojected vector a_(i)^(T)z_(p)

[0059] is tested against the projected mean a_(i)^(T)μ_(i)

[0060] for the respective class (the i^(th) class, in thisillustration).

[0061] To this end, a difference value d_(c) given by the expression:$\begin{matrix}{d_{c} = {{{a_{i}^{T}z_{p}} - {a_{i}^{T}\mu_{i}}}}} & (23)\end{matrix}$

[0062] is computed. If the test statistic d_(c) is greater than apredetermined threshold, t_(c) i.e. (if d_(c)≦t_(c)) authenticity of theprobe face image is accepted i.e. the claimed identity (that of thei^(th) client) is accepted; otherwise (i.e. d_(c)>t_(c)) authenticity ofthe probe face image is rejected i.e. the claimed identity is rejected.

[0063] The threshold value t_(c) is chosen to achieve a specifiedoperating point; that is, a specified relationship between falserejection of true claims and false acceptance of imposter claims. Theoperating point is determined from the ‘receiver operatingcharacteristics’ (ROC) curve which plots the relationship between thesetwo error rates as a function of decision threshold. The ROC curve iscomputed using an independent face image set, known as an evaluationset.

[0064] Typically, the operating point will be set at the ‘equal errorrate’ (EER) where both the false rejection and false acceptance ratesare the same.

[0065] In another approach, referred to herein as the ‘imposterrejection’ approach, the projected vector a_(i)^(T)z_(p)

[0066] is tested against the projected mean of imposters i.e.${a^{T}\mu_{\Omega}} = {{- \frac{N_{i}}{N - N_{i}}}a_{i}^{T}{\mu_{i}.}}$

[0067] To this end, a difference value d_(i) given by the expression:$d_{i} = {{{a_{i}^{T}z_{p}} + {\frac{N_{i}}{N - N_{i}}a_{i}^{T}\mu_{i}}}}$

[0068] is computed. In this case, the projected vector a_(i)^(T)z_(p)

[0069] for an imposter is expected to be close to the projected mean ofimposters. Thus, if the test statistic di is greater than apredetermined threshold t_(i) (i.e. d_(i)>t_(i)) the authenticity of theprobe face image is accepted i.e. the claimed identity (that of thei^(th) client) is accepted; otherwise (i.e. d_(i)≦t_(i)) authenticity ofthe probe face is rejected i.e. the claimed identity is rejected. Whenthe number of training face images N is large the mean of imposters willbe close to the origin and the second term in Equation 24 can beneglected. In this case, the difference value d_(i) will simply be theabsolute value of the projected vector a_(i)^(T)z_(p).

[0070] FIGS. 1(a) and 1(b) respectively show histograms of the teststatistic t, for probe face images of imposters and probe face images ofclients obtained using the ‘imposter rejection’ approach. As expected,the probe face images of imposters cluster at the origin, i.e. the meanof imposters in (FIG. 1(a)), whereas the probe face images of clientsfall far away from the origin (FIG. 1(b)). The negative projections inFIG. 1(b) are an artifact of the convention adopted for representing the‘fisher faces’. In principle, though, the projections of client faceimages could be computed to give a test statistic ti which is alwayspositive.

[0071] It has been found that the ‘client acceptance’ approach and the‘imposter rejection’ approach are complimentary and can be combined orfused. An example of the ‘fused’ approach is a simple serial fusionscheme. More specifically, a probe face image is initially tested usingthe ‘imposter rejection’ approach. If the probe face image fails thetest i.e. the claimant is rejected as an imposter, authenticity of theprobe face image is accepted. If, on the other hand, the probe faceimage passes the test i.e. the claimant is accepted as an imposter, theprobe face image is tested again using the ‘client acceptance’ approach.If the probe face image passes this second test i.e. the claimant isaccepted as a client, authenticity of the probe face image is accepted;otherwise, authenticity is rejected.

[0072] In this illustration, different threshold values t_(c) and t_(i)were used for the ‘client acceptance’ and ‘imposter rejection’approaches respectively. However, since the client and imposter probevectors z_(p) are both projected into the same one-dimensional space itshould be possible to find a common threshold value for both approacheswhich separate the client and imposter images at given operating pointerror rates.

[0073] The described personal identity authentication processes havebeen tested by conducting verification experiments according to theso-called Lausanne protocol described in a paper entitled “XM2VTSDB: TheExtended M2VTS Database” by K Messer et al Proc of AVBPA '99 pp 72-77,1999.

[0074] This protocol provides a standard framework for performancecharacterisation of personal identity authentication algorithms so thatthe results of different approaches are directly comparable. Theprotocol specifies a partitioning of the database into three differentsets; namely, a training set containing 200 clients, an evaluation setcontaining 200 clients and 25 imposters and a test set containing 200clients and 70 imposters.

[0075] The imposter images in the evaluation and test sets areindependent of each other and are distinct from the client set. Thetraining set was used to evaluate the client ‘fisher faces’, defined bythe transformations a_(i), as already described. The evaluation set wasused to determine the thresholds t_(i),t_(c) and the test set was usedto evaluate the false acceptance and false rejection rates onindependent data.

[0076] Prior to testing, the face images were correctly registered towithin one pixel to eliminate any contributory affects on performancedue to misalignment, and each image was photometrically normalisedeither by removing image mean or by histogram equalisation.

[0077] It has been found that when the ‘client acceptance’ approach isadopted the optimum results are obtaining using photometricallynormalised images, with histogram equalisation giving the best results,whereas when the ‘imposter rejection’ approach is adopted the optimumresults are obtained using images that are unnormalised.

[0078] The test results show that the ‘imposter rejection’ approachgives lower levels of false rejection/false acceptance than the ‘clientacceptance’ approach, and that even lower levels can be achieved usingthe ‘fused’ approach, these levels also being lower than levels that canbe obtained using conventional LDA authentication processes.

[0079] As already described, conventional LDA personal identityauthentication processes involve use of multiple, shared ‘fisher faces’spanning a sub-space having dimensions of 100 or more, necessitatingcomplex and computationally intensive matrix operations. In contrast tothis, the present invention involves use of a class-specific ‘fisherface’ defined by a transformation as which only occupies aone-dimensional sub-space. This has important implications forcomputational efficiency of the authentication process; morespecifically, because computational complexity in the operation phase(i.e. after the ‘fisher faces’ have been generated) is linearlyproportional to sub-space dimensionality, the class-specific approach ofthe present invention should operate at more than 100 times faster thanthe conventional approach.

[0080] Moreover, because the test statistics d_(c),d_(i) areone-dimensional there is no need to compute a Euclidean distance in‘fisher space’—a decision can be reached by a simple comparison of thetest statistic dad; with a threshold t_(c),t_(i), giving furthercomputational gains. Furthermore, the projections of client and impostermean$\left( {{a_{i}^{T}\mu_{p}} - {\frac{N_{i}}{N - N_{i}}a_{i}^{T}\mu_{i}}} \right)$

[0081] can be pre-computed and, again, this leads to faster processing.

[0082] Also, during the training phase, the class-specific ‘fisherfaces’ a can be evaluated in a relatively straightforward manner,without the need to solve an eigenvalue analysis problem. This isparticularly so when the number N of training face images is large;then, the between-class scatter matrix M_(i) tends to zero and thewithin-class scatter matrix $\sum\limits_{i}^{\quad}\quad$

[0083] simply becomes the mixture covariance matrix Φ which is common toall classes, giving yet further computational gains.

[0084] A further consequence of using a class-specific ‘fisher face’a_(i), as described, is that each such ‘fisher face’ can be computedindependently of any other ‘fisher face’. This makes enrollment of newclients relatively straightforward compared with the above-describedconventional LDA approach involving use of multiple, shared ‘fisherfaces’. Therefore, the present invention finds particular, though notexclusive, application in situations where the client population iscontinuously changing and the database of training face images needs tobe added to or updated.

[0085] The personal identity authentication process of the invention maybe implemented in a variety of different ways.

[0086] In a fully centralised personal identity authentication system aprobe face image is transmitted to a remote central processing stationwhich stores details of all the clients and carries out the necessaryprocessing to arrive at a decision as to authenticity.

[0087] Alternatively, a semi-centralised system could be used, as shownschematically in FIG. 2. In this case, class specific data; e.g. theeigenvectors v₁,v₂ . . . v_(m) and the mean vectors μ₁,μ₂ . . . μ_(m)are pre-computed and stored in a remote data store e.g. a portable datastore such as a smart card 10, and the data processing is carried out ina local processor 11. The local processor 11 stores the bases u₁, u₂ . .. u_(m) for the PCA projection matrix U and accesses data from the smartcard 10, as necessary, via a card reader 12. The processor 11 uses thisdata to process a vector z_(p) representing a probe face image receivedfrom an associated input unit 13.

[0088] In a yet further approach, a fully localised system could beused. In this case, all the necessary data is stored and processed in asmart card. With this approach, the present invention which involvesprocessing class-specific ‘fisher faces’ a_(i) should be m times moreefficient than the conventional LDA approach which involves processingmultiple, shared ‘fisher faces’, both in terms of data storage andprocessing speed.

[0089] Furthermore, enrollment of new clients in, or updating of thesmart card database, impractical using the conventional LDA approach,becomes feasible. Therefore, the present invention opens the possibilityof personal identity authentication systems having non-centralisedarchitectures.

[0090] In the foregoing implementations of the invention, the probe faceimage is presented as being that of one of m clients known to theauthentication system, and this image is tested against the respectiveclient class. In another implementation of the invention, the identityof the probe face image is unknown; in this case, the probe face imageis tested against one or more of the client classes with a view tofinding a match and establishing identity. This implementation findsapplication, inter alia, in matching of controlled photographs such asmug shots and recognition of suspects from CCTV video against a databaseof known face images.

[0091] It will be understood that the invention also embraces computerreadable media such as CD ROM containing computer executableinstructions for carrying out personal identity authentication processesaccording to the invention.

1. A personal identity authentication process comprising the steps of:using linear discriminant analysis (LDA) to derive a class-specificlinear discriminant transformation a from N vectors z_(j) (j=1,2 . . .N) defining respective training images, there being m different classesof said training images with the i^(th) class ω_(i) containing arespective number N_(i) of said training images such that${N = {\sum\limits_{i = 1}^{\quad m}N_{i}}},$

projecting a vector z_(p) defining a probe image onto saidclass-specific linear discriminant transformation a_(i), comparing theprojected vector a_(i)^(T)z_(p)

with a reference vector for the i^(th) class ω_(i) and evaluatingauthenticity of the probe image with respect to the i^(th) class independence on the comparison.
 2. A process as claimed in claim 1including, applying principal component analysis (PCA) to said vectorsz_(j) (j=1,2 . . . N) to generate a set of eigenvectors (u₁,u₂ . . .u_(n)), spanning an n-dimensional sub-space, as bases for a PCAprojection matrix U, using said PCA projection matrix U to project saidvectors z_(j) (j=1,2 . . . N) onto said eigenvectors (u₁,u₂ . . . u_(n))to generate respective n-dimensional vectors x_(j)(j=1,2 . . . N), andapplying said linear discriminant analysis (LDA) to said n-dimensionalvectors x_(j) (j=1,2 . . . N) to generate a class-specific eigenvectorv_(i) related to said class-specific linear discriminant transformationa_(i) by the expression a_(i)=Uv_(i).
 3. A process as claimed in claim 2wherein said step of applying said linear discriminant analysis (LDA) tosaid n-dimensional vectors x_(j) (j=1,2 . . . N) includes generating awithin-class scatter matrix $\sum\limits_{i}^{\quad}\quad$

for said i^(th) class ω_(i), and said class-specific eigenvector v_(i)is given by the expression: v_(i) = Σ⁻¹v_(i)

where$v_{i} = {\frac{1}{N_{i}}{\sum\limits_{j = 1}^{\quad N_{i}}x_{j}}}$

for all x_(j) in ω_(i).
 4. A process as claimed in claim 3 wherein saidwithin-class scatter matrix ∑_(i)  

is equal to the mixture covariance matrix Φ of said n-dimensionalvectors x_(j) (j=1,2 . . . N) where,$\Phi = {\frac{1}{N}{\sum\limits_{j = 1}^{\quad N}{x_{j}x_{j}^{T}}}}$


5. A process as claimed in claim 3 wherein said step of applying saidlinear discriminant analysis (LDA) to said n-dimensional vectors x_(j)(j=1,2 . . . N) includes generating a between-class scatter matrix M_(i)for said i^(th) class, where${M_{i} = {\frac{N_{i}}{N - N_{i}}v_{i}v_{i}^{T}}},$

and$v_{i} = {\frac{1}{N_{i}}{\sum\limits_{j = 1}^{N_{i}}\quad x_{j}}}$

for all x_(j) in ω_(i) generating the covariance matrix Φ of saidn-dimensional vectors x_(j) (j=1,2 . . . N), where$\Phi = {\frac{1}{N}{\sum\limits_{j = 1}^{N}\quad {x_{j}x_{j}^{T}}}}$

and generating said within-class scatter matrix ∑_(i)  

given by Σ_(i) = Φ − M_(i)  .


6. A process as claimed in any one of claims 1 to 5 wherein saidreference vector is defined as a_(i)^(T)μ_(i),  

where$\mu_{i} = {\frac{1}{N_{i}}{\sum\limits_{i = 1}^{N_{i}}\quad z_{i}}}$

for all z_(i) in ω_(i).
 7. A process as claimed in claim 6 wherein saidstep of evaluating includes evaluating a difference value d_(c) givenby: d_(c) = |a_(i)^(T)z_(p) − a_(i)^(T)μ_(i)|

and accepting or rejecting authenticity of the probe image in dependenceon said difference value.
 8. A process as claimed in claim 7 includingaccepting authenticity of said probe image if d_(c)≦t_(c) and rejectingauthenticity of said probe image if d_(c)>t_(c), where t_(c) is apredetermined threshold value.
 9. A process as claimed in claim 6wherein said step of evaluating includes evaluating a difference valued_(i) given by:$d_{i} = \left| {{a_{i}^{T}z_{p}} + {\frac{N_{i}}{N - N_{i}}a_{i}^{T}\mu_{i}}} \right|$

and accepting or rejecting authenticity of the probe image in dependenceon said difference value.
 10. A process as claimed in claim 9 includingaccepting authenticity of said probe image if d_(i)>t_(i) and rejectingauthenticity of the probe image if d_(i)≦t_(i), where t_(i) is apredetermined threshold value.
 11. A process as claimed in claim 7 andclaim 9 wherein said step of evaluating includes evaluating saiddifference value d_(i), accepting authenticity of said probe image ifd_(i)>t_(i), where t_(i) is a predetermined threshold value, evaluatingsaid difference value d_(c) if d_(i)≦t_(i), accepting authenticity ofthe probe image if d_(c)≦t_(c), and rejecting authenticity of the probeimage if d_(c)>t_(c) where t_(c) is a predetermined threshold value. 12.A process as claimed in claims 8,10 and 11 wherein said predeterminedthreshold value(s) t_(c),t_(i) are the same.
 13. A process as claimed inany one of claims 1 to 12 including deriving a respective saidclass-specific linear discriminant transformation a_(i) for each of saidm classes, and performing said steps of projecting, comparing andevaluating for at least one of the class-specific linear discriminanttransformations a_(i) so derived.
 14. A process as claimed in any one ofclaims 1 to 13 including enrolling a new class of training imagescontaining N_(m+1) training images as the m+1^(th) class and applyingthe process of any one of claims 1 to 13 to N vectors z_(j) (j=1,2 . . .N′) defining respective training images, where N′=N+N_(m+1).
 15. Aprocess as claimed in claim 14 including updating said class-specificlinear transformation a_(i) to take account of said new class.
 16. Apersonal identity authentication process comprising the steps of:providing a class-specific linear discriminant transformation a_(i)derived by applying linear discriminant analysis (LDA) to N vectorsz_(j) (j=1,2 . . . N) defining respective training images, there being mdifferent classes of said training images with the i^(th) class ω_(i)containing a respective number N_(i) of said training images such that$N = {\sum\limits_{i = 1}^{m}\quad {N_{i},}}$

projecting a vector z_(p) defining a probe image onto saidclass-specific linear discriminant transformation a_(i), comparing theprojected a_(i)^(T)z_(p)

with a reference vector for the i^(th) class ω_(i) and evaluatingauthenticity of the probe image with respect to the i^(th) class independence on the comparison.
 17. A process as claimed in claim 16wherein said class-specific linear discriminant transformation a_(i) isderived using the process steps defined in any one of claims 2 to
 5. 18.A process as claimed in claim 16 or claim 17 wherein said referencevector is defined as a_(i)^(T)μ_(i),

where$\mu_{i} = {\frac{1}{N_{i}}{\sum\limits_{i = 1}^{N_{i}}\quad z_{i}}}$

for all z_(i) in ω_(i), and said step of evaluating is carried out bythe process steps defined in any one of claims 7 to
 12. 19. A personalidentity authentication system comprising data storage and dataprocessing means for carrying out the process according to any one ofclaims 1 to
 15. 20. A personal identity authentication system forevaluating authenticity of a probe image with respect to one or more ofm different classes of training images, wherein said training images aredefined by respective vectors z_(j) (j=1,2 . . . N), there being a totalof N vectors z_(j), and the number of training images in the i^(th)class ω_(i) is N_(i) such that$N = {\sum\limits_{i = 1}^{m}\quad {N_{i},}}$

the personal identity authentication system comprising data storagemeans for storing a class-specific linear discriminant transformationa_(i), as defined in any one of claims 1 to 5, for each of said classesω_(i) (i=1,2 . . . m), and data processing means for accessing a saidclass-specific linear discriminant transformation a_(i) from said datastorage means, projecting a vector z_(p) defining said probe image ontothe accessed class-specific linear discriminant transformation a_(i),comparing the projected vector a_(i)^(T)z_(p)

with a reference vector for the i^(th) class ω_(i) and evaluatingauthenticity of the probe image with respect to the i^(th) class ω_(i)in dependence on the comparison.
 21. An authentication system as claimedin claim 20 wherein said reference vector is defined as a_(i)^(T)μ_(i),

where $\mu_{i} = {\sum\limits_{i = 1}^{N_{i}}\quad z_{i}}$

for all z_(i) in ω_(i), and is also stored in said data storage meansand is accessible by said data processing means.
 22. An authenticationsystem as claimed in claim 21 wherein said step of evaluating includesevaluating a difference value d_(c) given byd_(c) = |a_(i)^(T)z − a_(i)^(T)μ_(i)|,

and accepting or rejecting authenticity of the probe image in dependenceon said difference value.
 23. An authentication system as claimed inclaim 22 including accepting authenticity of said probe image ifd_(c)≦t_(c) and rejecting authenticity of said probe image ifd_(c)>t_(c), where t_(c) is a predetermined threshold value.
 24. Anauthentication system as claimed in claim 21 wherein said step ofevaluating includes evaluating a difference value d_(i) given by$d_{i} = \left| {{a_{i}^{T}z_{p}} + {\frac{N_{i}}{N - N_{i}}a_{i}^{T}\mu_{i}}} \right|$

and accepting or rejecting authenticity of the probe image in dependenceon said difference value.
 25. An authentication system as claimed inclaim 24 including accepting authenticity of said probe image ifd_(i)>t_(i) and rejecting authenticity of said probe image ifd_(i)≦t_(i), where t_(i) is a predetermined threshold value.
 26. Anauthentication system as claimed in claim 22 and claim 24 wherein saidstep of evaluating includes evaluating said difference value d_(i),accepting authenticity of said probe image if d_(i)>t_(i), where t_(i)is a predetermined threshold value, evaluating said difference valued_(c) if d_(i)≦t_(i), accepting authenticity of the probe image ifd_(c)≦t_(c) and rejecting authenticity of the probe image ifd_(c)>t_(c), where t_(c) is a predetermined threshold value.
 27. Anauthentication system as claimed in any one of claims 20 to 26 whereinsaid data storage means and said data processing means are located atdifferent sites.
 28. An authentication system as claimed in claim 27wherein said data storage means is portable.
 29. An authenticationsystem as claimed in claim 28 wherein said data storage means is a smartcard or the like.
 30. An authentication system as claimed in any one ofclaims 20 to 29 wherein said data storage means stores precomputed data.31. A computer readable medium containing computer executableinstructions for carrying out the steps defined in any one of claims 1to
 18. 32. A personal identity authentication process as hereindescribed with reference to the Figures of the accompanying drawings.33. A personal identity authentication system as herein described withreference to the Figures of the accompanying drawings.
 34. A computerreadable medium as herein described with reference to the Figures of theaccompanying drawings.